1. Field of the Invention
The present invention relates to an image processing apparatus and its control method for inputting graphics data and moving image data, composing these data and outputting composite data.
2. Description of the Related Art
As a technique for composing graphics data with moving image data and outputting composite data, the following two types of methods are known.
(1) Obtaining moving image data by frame, generating instantaneous still image data, and rendering the still image data as a part of graphics elements with graphics data to be composed.
(2) Separately performing processing on moving image data and graphics data with a moving image data processing system and a graphics data processing system, and composing data processed by the respective processing systems.
Video editing software operating on a computer is known as representative software based on the method (1). Further, as a technique based on the method (2), Japanese Patent Laid-Open No. 06-335022 (D1) discloses a technique in which a graphics generator that processes a drawing command and a video data generator that outputs video data are separately provided, and the outputs of the graphics generator and the video data generator are switched based on transparent data included in the drawing command. Further, Japanese Patent Laid-Open No. 07-72850 (D2) discloses a technique of separately providing a moving image frame buffer and a graphics frame buffer and combining graphics data with moving image data under the control of a window system. Further, Japanese Patent Laid-Open No. 2002-222428 (D3) discloses a display apparatus provided with separate graphics generator and video processor. According to the document D3, the graphics generator generates graphics data from a drawing command, generates video information (area information and translucent information) included in a graphics area and holds the generated data. On the other hand, the video processor performs video reduction processing using the video information. Then a composer performs translucent composition on the graphics data and the video data.
Further, Japanese Patent Laid-Open No. 10-020850 (D4) discloses a superimposing apparatus for superimposing video data and graphics data and displaying the superimposed data. According to the document D4, in an area where the graphics data and the video data overlap each other, priorities of the graphics data and the video data are selectively designated, thereby superimposed display of the video data and the graphics data can be realized at a low cost.
Further, Japanese Patent Laid-Open No. 2002-229767 (D5) discloses a moving image reproducing apparatus for reproduction of combined display of plural moving images and a drawn image. According to the document D5, in an environment where plural software operate, a display is produced by some software over a display produced by other software. A frame image is drawn in accordance with shape setting designation information of a moving image to be reproduced, thereby superimposition of a non-rectangular image is realized.
Further, Japanese Patent Laid-Open No. 2001-184523 (D6) discloses a technique of composing moving image and moving object while changing the order of superimposition. According to the document D6, a depth value indicating the priorities for front-display of the moving image and the moving object is applied, and shading is performed based on the depth value.
Further, Japanese Patent Laid-Open No. 2003-338991 (D7) discloses an image display apparatus for performing resolution-conversion on moving image data and graphics data in different resolutions and composing the data. According to the document D7, a moving image conversion unit and a graphic conversion unit respectively perform resolution conversion before composition of the moving image data and the graphics data.
However, in the above-described conventional techniques, it is impossible to compose graphics data and moving image data and output the composite data while perform general rendering processing applied to still image data (deformation, rotation, transparent processing and the like) on the moving image data, in real time.
In the above-described method (1), various types of rendering processings can be applied to moving image data, while these processings cannot be performed in real time. On the other hand, in the above-described method (2), realtime processing can be ensured, however, rendering processing almost cannot be freely performed on moving image data. That is, conventionally, it is impossible to ensure both freedom and real time processibility in rendering processing.
Further, in the above-described method (1), functional problems can be solved by introducing a very fast drawing processor. However, since such processor is very expensive, it cannot be incorporated in an apparatus for consumers.
Further, when an image including moving image and graphic image is displayed, to display the moving image in a graphics-drawn frame, it is necessary to display graphics data and moving image data in synchronization with each other. However, in an image display apparatus provided with separated moving image and graphics processing systems, time for completion of processing of moving image data and time for completion of processing of graphics data cannot be synchronized with each other. Accordingly, 1:1 binding between graphics and moving image frames cannot be realized.
Regarding this problem, in the moving image display apparatus disclosed in the document D2, only the position of composition is designated using an address management information selection unit for a frame memory. Accordingly, the positions of moving data to be composed and image data can be designated, however, a particular moving image data frame and a particular still image data frame cannot be composed. Further, a drawing circuit disclosed in the document D4 independently generates identification bits of video and graphics data with a graphics identification circuit connected only to the drawing circuit and a video identification circuit connected only to a video pixel generation circuit. Accordingly, processing times in the drawing circuit and the video pixel circuit cannot be synchronized with each other. Further, in the moving image reproduction method disclosed in the document D5, a window system is used for setting of shape information of a moving image and control of a drawing unit to draw a frame image, by each frame of the moving image. Accordingly, overhead occurs in the window system by frame, thereby the moving image reproducing speed is limited. Further, when image processing completion time in the drawing unit and that in the shape setting unit do not correspond with each other, the shape setting information and the result of drawing cannot be displayed in synchronization with each other.
Further, in the image generation system disclosed in the document D6, when a moving object and movie data are composed in other frame than a final frame of the movie data, time for moving processing of the moving object is not considered. Accordingly, if the movie data which is operating and the object after moving processing are composed, a video frame in which route information is designated and the composed moving object are shifted. Further, in the image display apparatus disclosed in the document D7, the graphic conversion unit and the moving image conversion unit respectively perform conversion processing and the results of conversion are composed and displayed. Accordingly, if the image processing by the moving image conversion unit and that by the graphic conversion unit are not completed simultaneously, the converted moving image data and the graphics data cannot be displayed in synchronization with each other. Further, in the image display apparatus disclosed in the document D3, superimposition between an image of a part to be composed in translucent state and a video image is designated using a setting unit. In the document D3, if the processing of the image of the part and that of the video image are not completed always with the same latency, then the image of the part and the video image cannot be composed and displayed in synchronization with each other.